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Abstract 

It is well known that Shannon's rate-distortion function (RDF) in the colored quadratic Gaussian 
(QG) case, can be parametrized via a single Lagrangian variable (the "water level" in the reverse water 
filling solution). In this work, we show that the symmetric colored QG multiple-description (MD) RDF 
in the case of two descriptions, can be parametrized via two Lagrangian variables. To establish this 
result, we use two key ideas. First, we propose a new representation for the MD test channel, and show 
that the mutual information rate across this channel coincides with the QG MD RDF. Second, we use 
variational calculus to obtain a spectral domain representation of the test channel's optimal side and 
central distortion spectra given the source spectral density and the side and central distortion constraints. 
The distortion spectra are specified via two Lagrangian parameters, which control the trade-off between 
the side distortion, the central distortion, and the coding rate. We also show that the symmetric colored 
QG MD RDF can be achieved by combining noise-shaped predictive coding, dithered quantization, 
and memoryless entropy coding. In particular, we show that the proposed MD test channel can be 
materialized by embedding two source prediction loops, one for each description, within a common 
noise shaping loop whose parameters are explicitly found from the spectral-domain characterization. The 
source prediction loops exploit the source memory, and thus reduce the coding rate. The noise-shaping 
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loop controls the trade-off between the side and the central distortions, by shaping the quantization 
noise. 

Index Terms 

Multiple-description coding, rate-distortion theory, predictive coding, noise shaping, delta-sigma 
quantization, optimization, calculus of variations. 

I. Introduction 

The traditional multiple-description (MD) problem (H, considers a source, which is en- 
coded into two descriptions that are transmitted over separate channels. Either one of the channels 
may break down and thereby cause a description loss at the receiver. The decoder knows which 
channels that are working but the encoder does not. The problem is then to design the two 
descriptions so that they individually represent the source to within some desired distortion level 
and yet are able to refine each other. Thus, combining the descriptions improves upon their 
individual performances. In the symmetric situation, the two descriptions are balanced, i.e., they 
are encoded at the same coding rate, and leads to the same distortion when used separately at 
the decoder. 

The achievable rate-distortion region for the MD problem is only completely known for very 
few cases. El-Gamal and Cover 0]| presented an achievable rate region for two descriptions 
and memoryless sources and Ozarow [|2l showed that in the white quadratic Gaussian (QG) 
case, i.e., for white Gaussian sources and MSE distortion, the El-Gamal and Cover region is 
tight. An achievable rate region for the case of general (time-correlated) stationary Gaussian 
sources and mean-squared error (MSE) distortion was recently characterized by Chen et al. J3). 
In particular, it was shown in [|3l that the achievable rate region forms a closed and convex set 
and that the minimal description rates can be found by extremizing over all distortion spectra 
satisfying the individual side and central distortion constraints. No explicit solution to the optimal 
distortion spectra was found. However, some intuition towards a spectral domain characterization 
was provided. Specifically, it was shown that the optimal rates for stationary Gaussian sources 
can be expressed as the sum of rates of parallel channels, each one representing a frequency 
band. Each of the channels must be tuned to a minimum Ozarow MD rate for some frequency 
dependent distortion level. In some sense, this can be seen as a reverse "water-filling" approach 
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where instead of having a flat water level as in the conventional single-description (SD) case, 
the water level is frequency dependent. The authors also pointed out that obtaining an explicit 
spectral domain solution from their results is technically non-trivial. Instead it was argued that 
the optimal rates can be found through numerical optimization by approximating the source 
spectral density by piece-wise constant functions. However, in general, for arbitrarily shaped 
sources, this becomes an infinite-dimensional optimization problem. 

In this paper, we present a parametrization of the symmetric colored QG MD rate-distortion 
function (RDF). While Shannon's RDF in the SD case can be parametrized by a single Lagrangian 
variable [4j (usually referred to as a "water level"), we show here that the symmetric colored 
QG MD RDF can be parametrized via two Lagrangian variables!] To establish this result, we 
use two key ideas. We propose a new representation for the MD test channel (see e.g., Fig. IT3T) . 
and show that the mutual information rate across this channel coincides with the QG MD RDF. 
Moreover, the mutual information rate is shown to be equal to the scalar mutual information 
over an additive white Gaussian noise (AWGN) channel, and the test channel can therefore be 
realized with white Gaussian quantization (e.g., high dimensional lattice quantization). This still 
leaves open the optimization of the spectral parameters, which minimize the mutual information 
rate. Instead of taking the conventional approach of diagonalizing the colored Gaussian source 
and thereby obtain an infinite number of independent sources (which might result in an infinite- 
dimensional optimization problem) we show that it is possible (and feasible) to directly optimize 
over the continuum of the test channels' side and central distortion spectral densities through 
the use of calculus of variations [|5]|. The resulting distortion spectra are then specified via 
two Lagrangian parameters, which control the trade-off between the side distortion, the central 
distortion, and the coding rate. Thus, we avoid extremizing over all functions representing 
admissible distortion spectra subject to the two distortion constraints. Instead, our results reveal 
that this otherwise intractable infinite-dimensional optimization problem, can be cast as a two- 
dimensional optimization problem over two non-negative Lagrangian parameters subject to the 
same distortion constraints. 

In 0, it was shown that Ozarow's white Gaussian MD RDF can be achieved by noise-shaped 
coding based on dithered Delta-Sigma quantization (DSQ) and then followed by memoryless 

'in our case, however, the two parameters cannot generally be interpreted as "water levels". 



September 29, 2011 



DRAFT 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, MAY 2010 



4 



entropy coding. Furthermore, by exploiting the fact that Ozarow's test channel becomes asymp- 
totically optimal for stationary sources in the high-rate regime [7], it was shown in [0 that, at 
high resolution, the stationary MD RDF is achievable by DSQ and joint entropy coding. In (31 
it is demonstrated how one can achieve any point on the boundary of the colored Gaussian 
achievable rate region by a frequency-domain scheme, where the source is divided into sub- 
bands, and in each sub-band the "quantization-splitting" scheme for white Gaussian sources 
presented in [8] is applied. 

In this paper, we propose a time-domain approach: We show that the symmetric colored 
QG MD RDF can be achieved by noise-shaped predictive coding and memoryless dithered 
quantization (in the limit of high dimensional quantization) at all resolutions and all side-to- 
central distortion ratios. We establish this result by forming a nested prediction / noise-shaping 
structure containing a dithered DSQ scheme similar to fl6] in the outer loop and a predictive 
coder per each description in the inner loop, see for example Fig. [14] Each of the predictive 
coders has the structure of the differential pulse-code modulation (DPCM) scheme, shown to be 
optimal in the SD setting in [9]j^ The role of the DSQ loop is to shape the quantization noise 
so that a desired trade-off between the side distortions and the central distortion is achieved. 
It was shown in [6J that the central distortion is given by the power of the noise that falls 
within the in-band spectrum (i.e. the part of the frequency spectrum which overlaps the source 
spectrum) whereas the side distortion is given by the power of the complete noise spectrum, 
i.e. the in-band and the out-of-band noise spectrum. It was furthermore shown that any ratio 
of side-to-central distortion can be obtained by proper shaping of the quantization noise. We 
establish a similar result here. In particular, the predictive coders exploit the source memory and 
thereby minimize the coding rate and make sure that memoryless entropy coding is optimal. 
Moreover, the DSQ loop performs the noise shaping, which is required in order to achieve any 
desired pair of side and central distortions. At general resolutions, the optimal noise shaping 
depends upon the source spectrum. However, at high resolutions, the optimal noise shaping 
becomes independent of the source spectrum and converges to a piece-wise constant function 
with a single jump discontinuity. 

2 The idea of exploiting prediction in MD coding has previously been proposed by other authors, see for example the following 
related works [10]-[13|. All these works faced the basic problem: Since DPCM uses prediction from the reconstruction rather 
than from the source itself, and this prediction should be reproduced at the decoder, it is not clear which of the possible 
reconstructions should be used for prediction. The present work solves this problem. 
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This paper is organized as follows. In section HH we provide the preliminaries. Then, in 
Section |inj we propose a test channel, which provides a new interpretation of the QG MD RDF. 
We derive the optimal distortion spectra in Section [TV] With the test channel in mind, we present, 
in Section |Vj an SD time-domain scheme which encodes a source subject to a distortion mask. 
Then, in Section |VH we extend the SD time-domain scheme of Section |V] to the MD case. 
Conclusions are in Section IVIII Longer proofs are deferred to the appendices. 



A. Stationary Gaussian Processes and Spectral Decomposition 

Let X = {X[n]}^L be a zero-mean discrete-time stationary autoregressive Gaussian process 
with power spectral density Sx = {Sx(e? u )}Z=-Tr We assume that Sx obeys the Paley-Wiener 
conditions |fl4|. such that it has a positive entropy-power < P e (Sx) < oo, where the entropy 
power of a spectrum Sx is defined as P e (Sx) — ~^r e e2hlyX ' '■ I n tne stationary Gaussian case, 
h(X) = ilog(27re) + ^ J^ n \og(Sx(e juJ ))du and the entropy power is therefore given by 



where here and onwards all logarithms are taken to the natural base unless explicitly stated 
otherwise. The source may be represented in the time-domain by 



II. Preliminaries 




(1) 



oo 




(2) 



fc=i 



or in the ^-domain by 



1 



(3) 



1 - A(z) 



where {/[n]} 



n=0 



oo 



is i.i.d. zero-mean Gaussian and where 



oo 




(4) 



i=l 
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is the optimal predicted associated with S X - Using this notation, a spectrum S x has a spectral 
decomposition [fT51 : 

Pe(S X ) 



S x (e 3U 



(1 - A(z))(l - A* (£)) 



(5) 



B. Symmetric Quadratic-Gaussian MD RDF 

Let // fc) be the encoder for the ith description, i.e., {^[n]}^ = /f° ({X[n]} k n z} ) and let gf } 
denote the zth decoder, i.e., {Xj[n]}*=Q = 0i ({^[w]}*=q). Similarly, let g c be the joint decoder 
that produces the central description {X c [ra]}^~Q. We will use the time-averaged mean squared 
error (MSE) as fidelity criterion. 

A rate pair (Ri, R 2 ) is said to be achievable with respect to a distortion triplet (Z?i, D 2 , Dc) 

(k) 

if there exists encoding functions f- ,i = 1,2, where for all sufficiently large k, 

ilogl/^l <Ri, z = l,2, (6) 

(k) (k) 

and there exists decoding functions gc ,g\ ,i = 1,2, such that the side and central distortions 
satisfy 

- fe-i 

lim T J2E[(X[n}-Xi[n}) 2 } < A, i = 1,2, (7) 

n=0 

and 

1 fc-i 

lim - Ve[(I[»] - X C H) 2 ] < 7J C . (8) 
fc->oo A; z — ' 



n=0 



In this work, we are interested in the symmetric case, where R 1 = R 2 = R and D 1 = 
D 2 — D$- The symmetric MD RDF is defined as the minimum rate R per description, which is 
achievable with respect to the distortion pair (Ds,Dc). When the source is white, we replace 
the spectrum Sx by the variance o\. In the colored case, we refer to R(e^ UJ ) as the rate spectral 
density, i.e., the rate required for coding Sx{e? u ) at some frequency u E [— ir;ir]. Similarly, 
we use the notations Dg{e^) and Dc{e^) for the side and central distortion spectral densities, 
respectively. With a slight abuse of notation, D s {e^) either refers to the complete spectrum 
{D 's(e J " a, )}£ = _ 7r or to the scalar distortion value at a single frequency to. Moreover, D s either 

3 We use the term optimal predictor to denote the unique filter, which when used to predict the source from its infinite past, 
minimizes the variance of the prediction error. If the source is Gaussian, the prediction error, i.e., {/[n]}, is a white Gaussian 
process I15I . 



September 29, 201 1 



DRAFT 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, MAY 2010 



7 



refers to the side distortion or to the side distortion constraint. In all cases, the meaning should 
be clear from the context. 

A high-resolution MD RDF is the limit where the coding rate diverges to infinity, and where 
the side and central distortions both tend to zero. Thus, under high resolution conditions, we 
have that S x (e juJ ) > D c (e juJ ) and S x (e> u ) > D s {e? w ), for all u G [-tt; tt] where S x (e juj ) > 0. 

C. Additional Notation 

For x real or complex,-^ has n roots. For n = 2 and < x G K we define ^fx = \\[x\, i.e., 
it is always non-negative. For > x G M we define y/x = z|yfz||, i.e., we take the principal 
complex root. For n = 3 and iGRwe lettfx = sign(x) \\/\x\\ denote the unique real cubic 
root of x, e.g., = —2. If x G C and imag(x) ^ 0, we let \/x denote the principal complex 
root, i.e., it has a positive imaginary part. We use the notation £f to indicate the ith root of the 
function 5. If cp is a function of we use the notation <£>|f=A to indicate that the function cp is 
evaluated at the point ( = A. If $ is a matrix, we use |$| to denote the determinant of $. 

III. The Quadratic Gaussian Symmetric MD Rate Revisited 

In this section we re- state known results about the QG MD achievable rate in the symmetric 
case, in order to gain some insight and prepare the ground for what follows. In the high resolution 
limit, these results also hold for general sources with finite differential entropy rate [16J. We 
consider four cases: we start with a white source, moving from high resolution to general 
resolution, and then we repeat the same for colored sources. 

A. White Source, High Resolution 

Let us first define the following rate expression 

R wh Ue,HR(°X, D C , D S ) = \ log ( - ^ = ] , (9) 

\ l y u c\L>s — l>c) ) 

which is only a function of the source variance and the side and central distortions and which 
is valid when the distortions are non-degenerate, i.e., for distortions satisfying^ 

2D C <D s <a\ (10) 

4 Of course, in order for the high-resolution approximation to hold, it must be that Ds <C Still, one gets a valid expression 
as long as Ds < o\. 
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and 



D c >2D s -a 



(ID 



In the symmetric rate/side-distortions case the quadratic-Gaussian MD RDF of Ozarow [|2l 
converges to ® in the high-resolution limit where o\ 3> max{Dc, D s } and when (fTOl) and 
(fTT|) are satisfied. 

In the high-resolution limit, if the central decoder was to linearly combine two side descriptions 
of mutually independent errors of variances D s , it would achieve exactly the maximal central 
distortion Dc = D$/2. This motivates the model of negatively correlated side noises (see 0). 
Indeed, in this limit, the relation between the side and central noises can be explained by the 
side noises having a correlation matrix: 



1 P 
P 1 



(12) 



where 



With this notation, © becomes: 



Da - 2D 



c 



D, 



< 0. 



R w hite,HR((? 2 x ,Dc,Ds) = - log 





1 , ( °x 
- log — 

2 ^\D S 



(13) 
(14) 
(15) 



where § HR is the high-resolution excess rate [16]. Still in the high-resolution case, we take 
another step: Without loss of generality, we can represent the correlated noises as the sum of 
two mutually independent noises, one is added to both branches while the other is added to one 
branch and subtracted from the other, as depicted in Fig. [Q In the figure, X\ and X 2 refer to 
the two side descriptions, and Xc refers to the central reconstruction. Note that the averaging 
eliminates Z_ from the central reconstruction X c . If we denote the variances of the noises Z + 
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Fig. 1. A differential form of Ozarow's double-branch test channel for high resolution coding. 



and Z_ as + and 0_, respectively, then we can re- write (fl"2l) as: 



with 



0+ + 0_ 0+ - 0_ 
©+-©_ 0+ + 0. 

|$| = 40 + 0_. 



(16) 



(17) 



Further, the distortions are related to these variances by: 

D s = + + 0_, 
D c = ©+. 



(18) 



Substituting in (fTTT) . we find that the non-degenerate distortion pairs correspond to the region: 

'2 



< 0+ < 0_ < 
_ + _ _ 2 

(which, on account of (fT6l) , confirms that only negative correlation is considered). 
Definition III.l. For variance o\ and 2x2 matrix $, the Rq function is defined as: 

^ log ("^f 



(19) 



(20) 



The following proposition gives the high-resolution optimal rate, based on the observations 
above. 
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Fig. 2. Ozarow's test channel with pre and post factors. 



Proposition 1. Parametric representation of R w hite,HR.(ox' Ds, D c ). For a white Gaussian 
source of variance o\, the high-resolution white MD RDF for all non-degenerate distortion 
pairs is parametrically given by varying (0 + , 0_) over the range (fl9l) and setting: 

1) Rwhite,HR(o'xj As, A?) = ^o(°x> ®)> wnere Ro is given by (l20l) and $ is given by (fT6l) . 

2) (D5, D c ) according to (fl"8l ). 



5. WTufe Source, General Resolution 

Generalizing our view to all distortion levels, the minimum achievable symmetric side-descriptions 
rate was given by Ozarow Q: 

R ( 2 n n \ a 1 / <4(<4-A C ) 2 \ , on 

i2wWto(a " D °' Ds) = I bg \AD C {D S -Dc){a\-Ds) ) ^ 

for all non-degenerate distortion pairs, which are the ones that satisfy: 

D s < o\ 

D c > 2D s -a\ 

^ > (22) 

D c ~ D s a\ 

A similar correlated-noises model to (fl"2l) can be obtained by expressing p in a rather com- 
plicated form. However, we can greatly simplify such an expression by proper use of pre- and 
post-factors, as depicted in Fig. [2] by oig and a c . In a point-to-point scenario, it is convenient to 
make the pre factors equal to the post factors [fTTll . lfT8l . However, this is generally not possible 
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in MD coding because the optimal post-factors (Wiener coefficients) are different for the side 
and central reconstructions. We choose the pre-factor to be equal to the side post-factor. While 
this choice seems arbitrary, it will prove useful in the sequel. Thus we have: 




e+-e_ 



<4 



«c ± =W " A V 2 T^,"' - (23) 



Noting that between U and {Vi,V 2 ,Vc} we have exactly the high-resolution test channel of 
Proposition Q3 it is straightforward to show that under this choice, the distortions are given by: 

d s = e + + e_, 



where as before 6 + and 0_ are the variances of the noises Z + and Z_ in Fig. [2] Note that at 
high resolution conditions a\ 9_, so (T24l) reduces to (fl"8l) . The choice of factors we have 
made maintains the high-resolution rate expression at any resolution; the following is identical 
to Proposition [Q except for replacing the distortion equations (fl"8l) by (124)) . 

Proposition 2. Parametric presentation of R w hite(o"x5 Ds, D c ). For a white Gaussian source 
of variance a\, the white MD RDF for all non-degenerate distortion pairs is parametrically 
given by varying (0+, 0_) over the range (fT9l) and setting: 

1) R w hit e {a 2 x , D s , D c ) = Ro(&x-> ®)> where Rq is given by d20l) and $ is given by (fT6l) . 

2) (D s , D c ) according to (|24]>. 



Proof: To show that it is enough to consider the triangular region (1191) . substitute (124)) in 
22l) to assert: 

0+ + 6_ < a\ 



aie+ >2(e + + e_)-4 



4-©- > 
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Now the second and third equations can be written as: 



■2 



v 0_ ] (a 2 x -&+-&_) >0 



a 

2 



(0_-0 + )K-e + -0_)>o, 

respectively, which, on behalf of the first equation, proves that <J 2 x /2 > 0_ > + . Finally, from 
the first two equations it follows that if + < then also + + 0_ < 0, but since in this case 
D s < 0, it is not possible. Deriving the rate expression is straightforward, by substituting the 
distortions (EH) in (ED- ■ 

C. Stationary Source, High Resolution 

We first re-state the optimal rates in propositions [Hand [2] for white sources in terms of spectra. 
This will yield a single expression which is also valid for colored sources. We therefore define 
the following equivalent distortion spectrum (which plays a part in the time-domain solution 
even for white sources, see H): 

f 20 + , \u\ < f , 
Q(e juJ ) = I (25) 
20_, | < \u\ < 7i. 

The entropy-power (OQ) of this spectrum is given by: 



P e (0) = 2v^07= yW 

Definition III.2. For source spectrum Sxi^) and noise spectrum Q{e^ w ), the Rq function is 
defined as: 

R (S X} Q)±lhg(^d). (26) 



2 °VP e (0) 

Since the source is white with spectrum Sx{e^) = o"x,Va;, we have that P e (Sx) = cr x . 
Consequently, _R = Ro, where R is the optimal rate in Propositions Q] and [2] and given by (|2Q|) 
and _R is given by d26l) . 

We now turn to general (colored) stationary Gaussian sources. In the high resolution limit, it 
was shown in [7] that the minimum rate is given by Ozarow's rate © with the source variance a\ 
replaced by its entropy-power P e (Sx) ©• Recalling Proposition Q] and (|26l) . this leads directly 
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Z+[n] 


X[n] 




U[n] 
















Z+in] 



\ Vi[n] 


F*(e> u ) 




J 







V 2 [n] 



Fig. 3. Pre-post filtered test channel for a colored Gaussian source. 



Q + (e ju 
t 



\Vc[n] 


G*(ei") 









~X [n] 



~X 2 [n] 



Fig. 4. The triangular support region of Q-(e J ") and 0+(e J ") as given by (|28 



to the following. 

Proposition 3. Parametric presentation o/Rhr(Sx ) D s , D c ). For a stationary Gaussian source 
of spectrum Sx{^ u ), the high-resolution MD RDF for all non-degenerate distortion pairs is 
parametrically given by varying (0+, 0-) over the range (fT9l ) and setting: 

1) Rhr{.Sx, D S) D c ) = Ro{Sx,®), where R is given by (|26l ) using the spectrum Q{e^) 
given by ((251) . 

2) (D s , D c ) according to (|24|) . 



D. Stationary Source, General Resolution 

For general resolution, the achievable colored Gaussian MD rate region was characterized by 
Chen et al. [3J. We can re- write the result (in the symmetric case) in the spirit of the above 
exposition. To that end, we present in Fig. [3] a test channel, which may be seen as a colored 
version of the channel of Fig. [21 In that channel, the noise processes Z + and Z_ are stationary 
Gaussian, with spectra {Q + (e j2uj )} and {0_(e :,2w )}, respectively. In terms of these, we generalize 
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' 26+ (e^) , 
Q(e? u ) = < 26_ ^' 2 ( w -5 
^ 26_ (e j2 ( w+ 5 

We will be interested in spectra that satisfy (fT9l) at each frequency, i.e. 

S x (e> 



14 



, f < W < 7T, 



(27) 



vr < w < -f . 



< 0+(en < 0_(e JW ) < 



(28) 



The per-frequency resulting triangular support region is shown in Fig. |4j 
The factors as and etc of of Fig. [2] are replaced by filters that satisfy^] 

S x {e* u ) - 9 + (e**0 - 0_(e*") 



|F(e^)| 2 

|G(e*")| a 
ZF(e iw ) 



Sx(e*") 

Sx{^){Sx{^ u ) ~ Q+{eP u ) ~ e_(e^)) 



(S^(e*-)-e_(e*-)) 5 



ZG(e 



(29) 



where AF{e? w ) denotes the phase of F[e? w ). It directly follows, that the side and central distortion 
spectral densities are given by: 



D 3 (e*") 4 5 ii _ x (en = S i2 _ x (e^) = e + (e^) + e_(en 
D c (e J ) = S+_ x (e? ) 



(30) 



which reduces to (|24l) at any fixed frequency w. The distortion spectra are connected to the total 
distortion constraints (D S ,D C ) via: 

±J* D s {e? u )du = ±J*[e + (e? u )+e-{e? u )]du < D s 
In these terms, we have the following: 



5 Notice that the lowpass and highpass spectra of G> are formed by {0+(e-' 2 ")} and {0_(e-' 2 ")}, which are compressed 
versions (by a factor of two) of the spectra 0+ = {Q + (e JU ')}Z=- 7 v an d O- = {@ -(e 3W )}v = — K > respectively. 

6 Without loss of generality, one may take all filters to have real frequency response. However, the more general complex form 
allows more flexibility, e.g. some of the filters may be made causal. 
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Proposition 4. Parametric presentation of R(Sx,D s ,D c )- For a stationary Gaussian source 
of spectrum Sx{e^), the RDF for all non-degenerate distortion pairs is parametrically given 
by taking the lower envelope (in the rate axis) of the (R, D§, Dq) region formed by scanning 
all spectra Q + (e^) and 0_(e J ") satisfying (|28l) and setting: 

1) R = R (Sx, 0), where Rq is given by (1261) using the spectrum ®(e> u ) given by (1271) . 

2) (D s , D c ) according to (|3D) . 

Proof: In the symmetric case, Theorem 4 of Q reduces to: 

R(S X ,D C ,D S ) = mm . -L / R wh u e (S x (e ju ), D c {e?»), D s (e j0J )) <kj, 



where the minimization is carried over all distortion spectra satisfying ((311) . and where R w hite is 
the MD rate for the white problem formed at each frequency u. Now without loss of generality, 
one may restrict the minimization to spectra that are everywhere non-degenerate as in ((22)) . i.e., 

D s (e j n < S x (e j ") 

D c (en > 2D s (e? u ) - S x (e? u ) 

1 > (32) 



D c (e^) ~ D s {ei") S x {ef u , 

for all cu (since if at some frequency this does not hold, we may reduce the contribution of that 
frequency to the total distortion without increasing its contribution to the total rate). Scanning 
all spectra satisfying this, amounts to scanning all spectra {0 + (e ja; )} and {G_(e J ' w )} satisfying 
(T28)) . Now since at any fixed frequency to, the equation system (1301) is just (124)) . we have by 
Proposition [2] that 

R(S x ,Dc,D s )= mm . — / -log j=4== , ■ du, (33) 



{e+( e ^)},{e_(e^)} 2tt y_ 7r 2 2 v /0 + (e^)6_(e^) 

where the minimization is taken over all spectra satisfying ((311) and (128)) . On account of ((27l) . 
this completes the proof. ■ 
Unlike Propositions [TH3l where any choice of in the range (1T91) led to an optimal rate, 
here we have an optimization over the spectra. This result can be seen as follows: Given some 
distortion spectra, the test-channel at each frequency is the scalar test-channel of Proposition [2] 
(it can be shown that this is the optimal test channel under distortion-mask constraints, but the 
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proof for that is not included in this work). Then, an optimization needs to be carried out over all 
distortion spectra such that the total distortion constraints are satisfied. The rate is thus specified 
as the solution of an optimization problem; in the next section, we provide an explicit solution 
for the optimal distortion spectra {D s {e^ u ')} and {D c {e^)}, by minimizing (|33l ) with respect 
to the noise spectra 9 + (e JW ) and 0_(e- 7tJ ). 

Remark 1. In the high resolution limit, the optimal spectra + (e- 7aJ ), B_(e- JW ) become flat, thus 
Q{e^) becomes a two-step spectrum, as in Proposition\3\and in 

Remark 2. If X does not satisfy the Paley '-Wiener condition, i.e. P e (Sx) = 0, then the rate 
expression (T26l) is not well defined. In this case, we may use the following: For any e > 0, let 
SxX e ^) = max(S'x(e- 7 ^), e), Vw, and D e = ^ J"^ max(0, e — Sx{e? u ))doj. Then there exists 
some e > such that Proposition |?] holds with Sx, D s , and D c replaced by Sx e , As + D e , and 
D c + D t , respectively. 

IV. Optimal Distortion Spectra 

In this section we derive a spectral domain characterization of the optimal distortion spectra. 
From Proposition @] and in particular from (|33l) . it may be noticed that finding the RDF is 
equivalent to finding a pair of noise spectra 6+ = {0 + (e jfcJ )}^ = _ 7r and 0_ = {6_(e- ?aJ )}^ = _ 7r , 
which minimizes the description rate R subject to the two target distortion constraints D s and 
Dc, cf. (|3TI) . This constrained minimization problem can also be formulated as a Lagrangian 
unconstrained problem, which then provides a two-parameter characterization of the RDF. Specif- 
ically, we consider the problem of minimizing the functional 

i r 

J = — / Ldco, (34) 



where 



2tt 



-logf - 7=4^2 , ■ i ! +A,.(eurn + eurn) 



2 \2y/Q + {ei»)Q-(e?") 



(35) 



+ A 



2" 



'Sx(e^) - e_(e*") 

The scalar weights Ai and A2 are Lagrangian variables JH, |[T9l , which provide a trade-off 
between rate, side distortion, and central distortion. Intuitively, letting Ai w 0, A2 ~ 0, results 
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in a rate close to zero since this is equivalent to solving a minimization problem without any 
constraints (except that of a non-zero rate). On the other hand, letting X 1 3> 1 or A 2 S> 1 penalizes 
one of the distortions and corresponds to a high rate situation. In particular, if Ai ^> A 2 , then the 
side distortion is severely penalized and is therefore forced to be small. The central distortion 
is of less concern in this case. If A 2 ^> Ai, then the central distortion is minimized and the 
side distortion is of less concern. Finally, if both Ai, A 2 3> 1, then both the side and central 
distortions are small. From the Kuhn-Tucker theorem, it follows that the Lagrangian variables 
are non-negative, i.e. Ai, A 2 > 0, see ll20l for details. If either Ai = 0, A 2 > or Ai > 0, A 2 = 0, 
the optimization problem reduces to a conventional SD problem. If Ai = A 2 = 0, we have an 
unconstrained problem, with a trivial zero rate solution. Thus, without loss of generality we may 
assume that Ai > 0, A 2 > 0. 

Fortunately, the unconstrained minimization problem (1341) may be solved by optimizing the 
per-frequency function L, as we formally state in the sequel. To that end, we obtain the following 
partial derivatives of (1351) : 

+ Ai + A 2 - . . , . . , (36) 



dG + (e? u ) 49 + (e»' w ) S x (e j0J ) - G-(e? u ) 

and 

«■ 1 *<f>^ ( fl> • (37) 



96_(eJ' w ) 46_(e? w ) (S x (^ u ) - e_(e*")) 2 ' 

Equating both (l36l) and (ITTT) to zero and jointly solving, yields 

e_(e> w ) = *{ lM (en (38) 

and 

Q.(e? u ) = — - Al ' A2 - (39) 

45 x (^)(Ai + A 2 )-4A 1 *i 1)Aa (^)' 

where for a fixed pair (Ai, A 2 ) G M 2 , *&* x \ 2 { e ^) denotes a real (and positive) root of the third- 
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order polynomial 

*s (e *-) _ 4AiA 2 W») + 8XlS x (en + A^ ^ 

2\ 1 S x (e>") + 2\ 2 S x (e>") + 4A 1 A 2 S|(e^) + 4Af5|(e^) S|(e>Q(A 2 + AQ 

+ 4Af * (C } 4Af ' 

(40) 

Since (|40l) is a real third-order polynomial in \Er(e J ' w ), three solutions are possible (of which two 
might be complex conjugates). Given a real polynomial x 3 (e : ' ul )+a 2 (e :iui ) x 2 (e- 7W )+a 1 (e : ' w ) x(e- ?w ) + 
a (e JW ), where the aj(e JW )'s follow from (l40l) . we denote its discriminant H(e 3 ' w ) by 

S(e J ' w ) = g 2 (e 3 ' w )+p 3 (e J ' w ), (41) 

where 



3 9 
1 



144A? 
+ 16^(e^)A 2 2 + l) 



SXxSxi^) ~ 16A 2 S x (e J ") + 16A 1 A 2 5|(e^) + 16A 2 Si(e^) (42) 



and 



q(en = -Me"*) a 2 {en - 3a (e^)) - -^-± (43) 
o z7 

' / 96A 1 A 2 ^(e jaj ) - 48A?5i(e^) - U\\S x {e jw ) - 96A^|(e^)Ai 



1728A? 

+ 96A^Sf (e^) + 96A 2 5i(e^)A? + 24A 2 S x (e^) + 64A^|(e^) + 12A 1 5 x (e Ja; ) - 1 

(44) 



Moreover, let si(e J ' w ) =yq{e? u ) + a/-^) and s 2 (e iaj ) =^jq(e^) - y/E(e? u ). Then, the three 
solutions are given by (see e.g., rt2TTl ) 



zi(e*") = ( Sl (en + S2 (e*")) - (45) 
x 2 (e^) = -~(si(e^) + *,(«>)) - + ^{siien - s 2 (e^)) (46) 

^s(e^) = -±(ai(e*") + s 2 (e^)) - - ^(si(e^) - Sa (e*")). (47) 
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At each frequency u, the global minimum of L could be one of these solutions, or lie on the 
boundary of the region of support given by (1281) and shown in Fig. HI 

Lemma 1. For any —n < u < tt, A x > 0, and A > 0, there is at most one unique stationary 
solution within the triangular support region (|28l ). If such a solution exists, it is given by ^/(e^), 
where 

\x 2 (ei"), i/S(e*")<0, 
^{e JW ) = I (48) 

[x 1 (e> u ),ifE(e> u )>0, 

and where x\{e^ w ) and x 2 {e^) given by (1451 ) and (|46l) , respectively, can be explicitly expressed 
in terms of \\ and A 2 by 

Xl (en =tfq(ej") + ~ \j ^0 - V 7 ^) + (2 Ai + A 2 ) + ^- (49) 

and 

x 2 (en = -/b(^(cos(0(e^)/3) + V^sm(0(e^)/3)) + ^p(2Ai + A 2 ) + (50) 

The proof is given in Appendix [A] To resolve when the global minimum is given by (1481) and 
when it is on the boundary, we define the set fl(Ai, A 2 ) of support frequencies by: 

Q(\ u A 2 ) = {-tt < u < tt : 2X l S x {e 3U1 ) + 8A 2 ^(e iw ) > 1}. (51) 

Lemma 2. For any positive spectrum Sx, the triplet (co, X±, X 2 ), where —n < u < ir, Ai > 0, and 
A 2 > 0, yields a stationary solution ^/(e^) when used in (|48l . If oj e fi(Ai, A 2 ), ?/zen ^/(e- 7 ") w 
globally optimal. Otherwise, the globally optimal solution is Q + (e^) = 0_(e J ' u ) = Sx(^ 0J )/2. 

Proof: For any u E [—it; it), the boundary of the triangular optimization region (1281 (see 
also Fig. BJ is given by the union of the following three faces: 

1) e + (e>' w ) = 0,0 < 0_(e J ' w ) < 5 x (e- ?a; )/2. 

2) < 9 + (e^) = e_(e>' u ) < S x (e j ")/2. 

3) 0_(e 3 ' w ) = 3x{e iu )/2,Q< &+(e ju> ) < e_(e 3 " w ). 
We now consider the behavior of L near these faces. 

1) If 9 + (e^) = 0, then i? ->■ oo for any non-negative Ai, A 2 and Q.(e juJ ) < S x (e juJ ). Thus, 
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moving from the boundary and inside the region can only reduce the function. 
2) If < Q+(e? u ) = e_(e>' u ) < S x (e jul )/2 then D s (e juJ ) = 26+ (e* - ) and the spectral rate 
is R(e^) = ^ log ^g(e^) )' wri ich is optimal for the side descriptions, i.e., each side 
description is on the SD RDF. The (unnormalized) directional derivative, i.e., the "normal" 
vector to the surface Q + {e^) = 6_(e JW ), which points into the support region, is given 
by the following difference of partial derivatives: 
<9L <9L 



96_(e» w ) 0e+(e*") 

X 2 S x (e^) I 26+(e^) - S x (e^ 



(52) 



Sx(& u ) - 6+(e^) I S x (ef u ) - 6+(e> 



where we have used d36j) and (|37l) , substituting + (e JW ) = 6_(e jtJ )). Since Q + (e ju; ) < 
S x (e> u )/2 it follows that (|52|) is always negative. Thus, L can be reduced by going 
away from the boundary and into the support region, i.e., by letting < 6+(e- JU ') < 
6_(e^) < S x (e^)/2. When 6_(e^) = 6+(e^) = S x (e^)/2, the spectral distortions 
satisfy D s (e juj ) = D c {e> u ) = S x (e juJ ) and the spectral rate is R(e? u ) = 0. This is only 
optimal for Ai = A 2 = 0. 
3) For 9_(e- ?< ^) = S x (e jul )/2, the directional derivative is given by 



1 + A 1 + ^ft (53) 



dQ-(e? u )) 

which is positive along the whole boundary if and only if u £ 0(Ai, A2) (ISTI) . Thus for 
support frequencies the Lagrangian can be reduced by going away from the boundary 
and into the support region, while for other frequencies it is increased for some boundary 
points. 

Now we complete the proof, using that inside the region, L is differentiable, and that by 
Lemma Q] it has at most one externum, which is given by ^/(e jaj ). For the first part of the 
lemma, u G fi(Ai,A2). The function decreases by moving inwards from all boundary points, 
thus it must indeed have an externum, which must be a local and global minimum. We now prove 
the second part of the lemma. For non- support frequencies the function increases by moving 
inwards from part of the boundary 0_(e J ' w ) = S x (e juj )/2,0 < 6+(e^) < 0_(e^). Thus, if it 
has a single externum inside the region it cannot be a local minimum, and (regardless of the 
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existence of an extremum) the global minimum must lie on that boundary. Now recall that L 
is strictly convex in e + (e ju) ) for a fixed < 0_(e JtJ ) < S x (e ju ) and that Q + (e juJ ) = is a 
maximum. Moreover, 



dL 



e-(e^)=s x (e^)/2 40+(e- ? ' 



' f Ai + 2A 2 , (54) 



is zero at 6+ (e ju ) = l/4/(Ai+2A 2 ) and corresponds to a global minimum of £(6+ (e jLU ), S x (e j ")/2). 
Let e_(e* w ) = S x {e juJ )/2,Q + (e^) = l/4/(Ai + 2A 2 ) and solve for A 2 when equating (07) to 
zero, i.e., 

As = ASxW) ~ Y' (55) 

which implies that L is minimized for 6 + (e- JW ) = Sx(e : ' UJ )/2. Thus, there is a single global 
minimum at 6 + (e- ?u; ) = Q-(e? u ) = Sx(e^)/2, which corresponds to a trivial zero-rate point. 



Remark 3. Since the rate spectral density R{e^), which is given by 

R{en = l -lo g ( - ^ ), (56) 

is zero for ®_(e>") = 9 + (e j,a; ) = Sx(e Ja, )/2, it follows that the set of support frequencies 
^(Ai,A 2 ) 071) defines a frequency dependent "water level" below which no rate is allocated. 
See Figs. \6\ and \7\for an example where zero rate is allocated to a part of a source spectrum. 

We are now in a position to present the symmetric two-description RDF in a parametric form: 

Theorem 1. Let X be stationary Gaussian with spectral density Sx, and having finite positive 
differential entropy rate, i.e., < h(X) < oo, and finite variance o\. Then the symmetric 
quadratic two-description RDF of X, R(Sx, Ds, D c ), is given for all non-degenerate distortion 
pairs by taking the lower envelope (in the rate axis) of the (R, Ds, Dc) region formed by 
scanning all Ai, A 2 > and setting: 
1) R according to: 

~ 1 ru . ?m ) M (57 ) 

4W-» 5 I 2Ve + (eJ"")e_(ei") I 



September 29, 2011 



DRAFT 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, MAY 2010 



22 



2) (D$, Dq) according to (|3T| ). 
In these expressions, for u G fi(Ai, A 2 ), 



and 



and otherwise 



e_(e> u ) = V(e j ") (58) 



S Y (e ju; ) - ^(e ju) ) 

QJe ]U ) = 1 ' 1 ] (59) 

+ { ] 4S x (e^)(A 1 + A 2 )-4A 1 *(e^)' V ; 



6_(e JW ) = 9+(e 3 ") = ' 



2 

where for any Ai, A 2 > 0, ^/(e- 7 ") and fi(Ai, A 2 ) are gz'ven /Vy and (|5TI) . respectively. 

Proof: The constrained minimization problem of Proposition @] can be cast as an extended 
isoperimetric problem and solved using standard techniques of variational calculus JH, lfl9l . In 
particular, we may form the cost functional shown in (|34l) and use the fact that the functions 
6_(e- ?w ) and Q + {e^) which extremizes (|34l) must also satisfy 

dL dL 

0, ^ , = 0, (60) 



99+ (e**) S0_(e> , 

for all — 7r < w < 7r, where the Lagrangian L is given by (1351) . Moreover, even though the side 
constraints are defined as integrals of functions, it is enough to consider constant Lagrangian 
multipliers (Ai,A 2 ) [19, Thm.2, p.91]. Thus, for a fixed pair of positive Lagrangian variables 
(Ai,A 2 ), the optimal spectra {9+(e Ja ')}£__ 7r and {G_(e J ' w )}£ = _ 7r are determined by Lemma [2] 
by sweeping over — n < tu < n. The RDF at the distortion point (D s , D c ) is then obtained by 
choosing that pair (Ai,A 2 ), which results in the lowest rate when used in ([57]) and where the 
corresponding distortions (I3TI) satisfy the constraints (Ds,Dc). ■ 
To elucidate the behavior of the noise spectra Q_(e> u ) and Q + {e^) as a function of Ai and 
A 2 , we present the following results and examples. 

Proposition 5 (High-Rate Cases). For any to G [— 7r; tt), 

lim A!9_(e^) = - (61) 

Ai,A 2 — >oo 4 
Ai/^^0 
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Ai 

Fig. 5. High-rate convergence of (Ai + A2)0+(e : "*') and \iQ-(e? u ) as Ai — >• oo when 5 , x(e J ") = 1, A2 = 2 and any ui. 



and 

lim (A 1 + A 2 )e + (e^) = i (62) 

Al ,A2~*OC 4 

Ai/v^^O 

Proof: See Appendix EJ ■ 

Remark 4. 77ze convergence requirement of A i/yfX-z — >■ m Proposition\5\is a technicality needed 
in the proof. As shown in Fig. \5\ the limiting behavior of rff571) and rf62l) can a/so oe observed 
for small A 2 ana* /arge Ai. This shows that under high-resolution conditions, the optimum noise 
spectra are flat, independent of the source spectrum, and approximately given by 0_(e JtJ ) ~ 

and 9 + (e*») « 

Example 1. Le? f/ze source have a positive and monotonically decreasing spectrum given by 

S x (e juJ ) = cos(u;) + 1, < \u\ < 7T, (63) 

and shown in Fig. |6] Moreover, let the distortion constraints be Dc = 0.08 and Ds = 0.4. 

t/smg dze closed-form expressions for the noise spectra provided by Theorem [7] we have 
numerically performed a simple grid search over Ai and A 2 . As Ai and A 2 are varied, we 
compared the resulting side and central distortions given by rt?71) to the above mentioned 
distortion contraints Ds and Dc- The noise spectra that resulted in distortions closest to the 
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2- 
1.8- 
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1.4- 
1.2- 
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0.8- 
0.6" 
0.4- 
0.2- 



Fig. 6. Source Sx, noise G + , 6-, and distortion {fls(e 3 ")}, {fls^")} spectra for < |u| < 7T. 

constraints are shown in Fig® The spectra were obtained with Ai = 0.2380, and A 2 = 2.700, 
which resulted in D c = 0.0801 and D s = 0.4000. Moreover, when using these spectra in 071) 
the obtained per description rate is R = 0.7468 bits/dim. In Fig. \6\ we have also shown the 
resulting side and central distortion spectra. 

To better illustrate the trade-off between central and side distortions as a function of the source 
spectrum, we have shown the ratio log(©_(e J ' w ) / + (e JW )) as well as the ratio \og(D s(e J,a; ) / Dc{e?' 
in Fig. [7J Also shown in Fig. [7J is the sum-rate 2R(e^) allocated to each frequency band, where 
R{e^ u ) denotes the per description rate spectral density rt5f5l) . It may be noticed that zero rate 
is allocated for the part of the source spectrum, which lies below a certain threshold (as is also 
the case in conventional SD reverse water-filling). 

Example 2. For a given uj G [— 7r; tt] let Sx{&* u ) = 2 and let Ai = 3. Then the noise spectral 
density components 0_(e- JtJ ) and @ + (e JW ) are shown in Fig. ^(expressed in dB) as a function 
of A 2 in the range [0; 10]. For A 2 < 0.52, the discriminant H is positive whereas for A 2 > 0.52, 5 
is negative. We have indicated the switching point with circles in the figure. Notice that already 
at e.g., A 2 = 4, the approximation 4^^^) of (HHl — 10 log 10 (4(Ai + A 2 )) = —14.47 dB 
provides a good approximation of ® + (e> w ). Also shown in Fig. [8] are ©-(e^) and Q + (e^ w ) as 
a function of Ai G [0; 10] for fixed A 2 = 3 and Sx{^ u ) = 2. In this case, E < for all Ai. 



Sx(e^) 




e_(e JW 














) 
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2.5r 



1.5 



0.5 




■we-fe^ye+Ce 5 ."')) 

■log(D s (e^)/D c (e^)) 



2rr 
3 



Fig. 7. Noise spectra ratio \og(Q-(e JU ')/Q+(e :,u ')), and distortion spectra ratio \og(Ds(e ,w ) / Ds(e 3U) )) as a function of the 
source spectrum given by J63t . Also shown is the sum-rate spectral density 2R(e 3U ') (in this latter case, the y-axis represents 
bits/dim. instead of distortion ratios). 



a -e 



o -10 
-12 
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-16 



— e_ 




,Ai 
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— e+ 




,Ai 


= 3 


---e_ 




,A 2 


= 3 


—e+ 




,A 2 


= 3 




Fig. 8. Q~(e 3 ") and 0+(e J ") as functions of Ai and as functions of A2. The circles indicate when S switches from being 
positive to become negative, which corresponds to the cases given by d48t . A single frequency bin where Sx{e 3U) ) = 2 is 
considered. 



Notice that at \\ = 4, the approximation of f|67T), i.e., — 10 log 10 (4Ai) = —12.04 and d62l ). 
i.e., — 10 log 10 (4(Ai + A 2 )) = —14.47, provide good approximations of ' G_(e 3 ' w ) and + (e 3 ' w ), 
respectively. As Ai — >■ 0, 0_(e JW ) — » Sx(e^ u! )/2, and since A 2 > 0, f/ze rate w wsed entirely 
for reducing the central distortion. However, as Ai increases, more rate is spend on decreasing 
6_(e-' w ) with less emphasis on Q + (e? u ). 
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Ai or A2 



Fig. 9. Dc(e 3 ") and Ds(eP") as functions of Ai and as functions of A2. A single frequency bin where Sx(e Jul ) = 2 is 
considered. 




Fig. 10. @-{e? u ), 0+(e J "), Dc(e 3 ™), and Ds(e 3iJ ) as functions of Ai = A2. A single frequency bin uj where Sx(e 3 '* } ) = 2 
is considered. 



The side and central distortions D s {e^) and D c {e^) for the above 6_(e JW ) and + (e 3 ' w ) 
and given u are shown in Fig. |9] In Fig. [70] we have illustrated 0_(e JW ), 6 + (e JtJ ), Ds(e? u ), 
and Dc{e? w ) as functions of \\ = A2, and the corresponding description rate spectral densities 
R{e? u ), given by rf5o1) . are shown in Fig. 1771 
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Fig. 11. R(e 3 ") as a function of Ai, as a function of A2, and as a function of Ai = A2. A single frequency bin where 
Sx{e 3W ) = 2 is considered. 



V. Time-Domain SD Source Coding Subiect to a Distortion Mask 

We take a detour to a SD problem that is suggested by Proposition HI coding of a source subject 
to a maximum distortion mask {D(e J ' u )}, rather than subject to a total distortion constraint. 
That is, we wish to compress a stationary Gaussian source X such that the spectrum of the 
reconstruction error satisfies!] 



Sx-x( e3W ) < D(e jw ),V-n <cu<n. 



(64) 



Without loss of generality^, we assume that D(e> w ) < Sx(e* w ),Vw- It is easy to verify, that 
the minimum rate for this problem is given by (recall (l26l)): 



R\Sx,D 



1, Pe(S X/ 

2 x PAD 



(65) 



Fig. [I2l presents a time-domain test channel which achieves this rate. Motivated by the ratio 
of entropy powers (|65l) . we strive to achieve the optimal rate by the combination of source 
prediction in order to produce a prediction error of power P e (Sx), and noise shaping in order 



7 ln principle, there is no guarantee that the reconstruction error is stationary. The condition can be thought of this way: pass 
the reconstruction error through a bank of filters, and measure the MSE as in the regular definition in Section [TTJ now take the 
limit of narrow filters. 

8 Otherwise, there is just wasted allowed distortion which does not serve to reduce the rate. 
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Fig. 12. A DSQ/DPCM equivalent channel for SD coding subject to a distortion mask. 



to shape the white quantization noise of power P e (D) into the spectrum These two tasks, 
we perform by a DPCM loop flU and a noise-shaping loop [6], respectively. In this test channel, 
Z[n] is AWGN of variance P e (D) and A(z), which is given by (HJ), is the optimal predictor of 
the source spectrum Sxlj Moreover, the noise-shaping filter is given by: 

Q{z) 



C{z) 



l-Q(z) 



(66) 



where Q(z) is the optimal predictor (HJ) for the distortion mask {D{e^)}, i.e., for 



z = e 



D(e JU 



Pe{D) 



P e (D)\l + C(e 



(67) 



|l-Q(ei-)| 

Note that E[n\, the input to the noise-shaping filter, is equal to Z[n}. The pre-filter F{e? w 



satisfies: 



S x (e? u ) - D{e>") 



(68) 



Theorem 2. For all source and distortion spectra satisfying Sx(e^) > -D(e JtJ ), Vw and oo > 
Pe(Sx) > Pe(D) > 0, the channel of Fig. \T2\with the choices above, satisfies: 



S x _ x (en = D(en, < w < n, 



(69) 



with the scalar mutual information I(D[n); Y[n]) = R[ Sx, D ) of (I65T ). 



An alternative time-domain approach, is to accommodate for the distortion mask by changing the pre and post-filters. However, 
we choose the noise-shaping approach for the sake of extending this test channel to the MD setting. 

10 We assume that the optimal predictor A(z) for the source spectrum exists. If not, then we may use the procedure outlined 
in Remark [2] in order to construct a predictor, which satisfies the assumption. 
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Proof: 

V[n) - B[n] = Y[n) - D[n] = Z[n), 
thus it is a white process with variance P e (D). Now 

V[n] - U[n) = (V[n] - B[n\) * (1 + c n ) 

thus by ([67]) 

S v -v{e?») = D(e?"). 

By the choice of F(e^) this is also the spectrum of X[n] — X[n], which completes the proof 
of the first claim. For the second claim, straightforward analysis of the scheme shows that 

Y[n] = (1 - a n ) * [U[n] + (1 + c n ) * Z[n\). 

Using the independence of {Z[n]} from and the choice of F(e Ja; ), A(z) and C(z), one 

finds that Y[n] is white with variance P e (Sx)- On account of the Gaussian channel capacity, the 
scalar mutual information I(D[n]; Y[n}) follows (the scalar mutual information is not changed 
by the channel input D[n] not being white). ■ 

Remark 5. The above proof does not use information-theoretic arguments, and does not offer 
intuition as to why the structure or the choice of filters are good. The same result can also be 
derived using an alternate proof which shows that the instantaneous mutual information over 
the AWGN from D[n] to U[n] must equal the mutual information rate between the processes 
U[n] and V[n]. The interested reader is referred to /[9|/ and for examples of such proof. 

Remark 6. In the special case of a white distortion mask D, the constraint becomes (by 
the water-filling principle) equivalent to a regular quadratic distortion constraint. Indeed, the 
channel collapses in this case to the pre/post filtered DPCM channel of Much of the 
analysis there remains valid for this problem as well. In particular, we can construct an optimal 
coding scheme using the test channel of Fig. [72] substituting the AWGN for an ECDQ, and 
the scalar mutual information I(D[n];Y[n]) is also equal to the directed mutual information 
I(D[n] — > Y[n}). See the end of Section [Wl for more details. 
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Fig. 13. A DSQ/DPCM equivalent channel for MD coding of a colored source. We use the index n for sequences which are 
"running" at the source rate, and the index m when referring to the upsampled rate. 



VI. Optimal Time-Domain Colored MD Coding 

We now turn back to the original colored MD problem. Proposition |4] tells us that if we opti- 
mize for the spectrum 0(e J£J ), then the test-channel of Fig. [3] describes the optimal R(Sx, D Sl D c ) 
However, we would like a test channel that provides a recipe for a coding scheme, i.e., the 
additive noise elements should be replaced by quantizers. Unfortunately, the noises in Fig. [3] are 
colored, and furthermore they are correlated, making the task hard. In this section we present a 
test-channel with a single white additive noise, in the spirit of the SD channel of Section [Yl After 
proving that the channel gives the "right" rate-distortion tradeoff, we present a corresponding 
coding scheme. 

A. Test Channel 

Fig. [T3] shows the adaptation of the distortion-mask equivalent channel to the MD problem. 
Following J6]|, we combine interpolation by a factor of two (upsampling and perfect low-pass 
filtering) with the noise-shaping loop, forming a DSQ loop. A(z) is the optimal predictor © of 
the spectrum Sx{e* u )- It is worth pointing out that the predictor is applied on the noisy output of 
the quantizer. Thus, the DPCM loop uses pure predictions only in the limit of high resolutions, 
see flU for details. Note also that we apply an upsampled version of the source predictor, namely 
A(z 2 ). The DSQ loop, on the other hand, works in the upsampled rate and the noise-shaping 
filter C(z) is given by (|66|) for the spectrum Q(e^) given by (ITTT) . Recall from (T27l) that Q+(oj) 
and 0-(uj) now play the roles of "in-band noise" and "out of band noise". In the high-resolution 
case, the DSQ is independent of the shape of the source spectrum and uses a two-step spectrum 
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as in the white case, see []6l for details. The additive Gaussian noise Z\m\ is white with variance 

p e (<s>) = 2v/p e (e + )p e (e_). 

Finally, the filters F(e^ UJ ) and G(e JaJ ) are chosen according to (|29l) . The channel is completely 
characterized, then, by the choice of + (e^) and 0_(e JtJ ), which can be performed e.g. ac- 
cording to the optimization in Section [IV] We show that regardless of the optimization, for any 
choice of spectra that satisfy (|28l) . we have a rate (recall (l33l): 



fl( s„e + ,e_) = iiog „ ) = 1- f liog„ J^:},.^. (70, 



2 \2VP e (e + )P c (9_) ; 2t7_,2 °2V8 + (ei")9_(e^ 

Remark 7. For a w/zzYe source, A(z) = anJ the channel reduces to the DSQ MD test channel 
of Ml/, while for optimal side distortion, C(z) = 0, and the channel reduces to an upsampled 
version of the DP CM equivalent channel of [9]. 



Theorem 3. The channel of Fig. \T3\ with the above choice of filters, satisfies: 

Sx c _ x (en = D c (en, -tt < u> < tt, 

Sx s -x(en = D s (en, -tt < u < tt, (71) 

where the distortion spectra were defined in ft30\) . while the scalar mutual information I(D[m]; Y[m)) 
equals the rate (|70l) . 



Proof: The channel between [/[to] and V[m] is identical to the one between U[n] and V[n\ 
in Fig. d2l We can thus use the proof of Theorem |2] to assert: 

[/[to] - V[m] = (1 + c m ) * Z[to] (72) 

y [m] = (1 - a m ) * ([/[to] + (1 + cm) * Z[m]), (73) 

where a m corresponds to A(z 2 ), i.e., equals a m / 2 for even to and zero otherwise. For the distortion 
spectra, (1721) means that Sv-u{^ u ) = 0(e JW ). From here, we use the structure of rate conversion 
and low-pass filtering, in conjunction with (|27l) . to obtain: 
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1 



0{e j2uj ) + 0{e j2ul -^ 



6+(e^) + 6_(e >J ). (74) 



By the choice of F(e^ UJ ) and G(e^ u ), the distortion spectra follow. For the mutual information, 
using the independence of {Z[n]} from {^7[n]} and the choice of F(e JW ), A(z) and C(z), one 
finds that Y[n] is white with variance P e (Sx)- On behalf of the noise variance and the AWGN 
channel capacity, the proof is completed. ■ 

B. Coding Scheme 

We now present a coding scheme based on the optimal time-domain test channel. The un- 
derlying principle is that an additive Gaussian white noise element in the test-channel may be 
replaced by an entropy-coded dithered quantizer (ECDQ), such that the rate of the quantizer is 
the scalar mutual information over the AWGN, and the variance of the (additive) quantization 
noise is the AWGN variance. This has been shown in lfT8l| . and extended to test channels with 
feedback loops in Q. We give a short account of these results, before turning to the MD scheme. 

At all resolutions, an ECDQ can approach, at the limit of high lattice dimension K — > oo, a 
rate 

where a\ and <r| are the output and quantization noise variances, respectively. Furthermore, the 
quantization noise is independent of the quantizer inputLlj Thus, in conjunction with optimal 
factors (i.e. Wiener estimation), the white Gaussian RDF is achieved lfT8l . In the presence 
of feedback loops, it is convenient to assume the existence of a large number of identical 
and mutually independent sources, equal to the lattice dimension K. These sources are treated 
independently by the encoding/decoding scheme, except for the actual ECDQ which processes 
them jointly. Thus we will only present the scheme for one source, but the quantization noise 
has the properties of a high-dimensional ECDQ. In a realistic setting, the multiple-source setting 
may be emulated by a single source which is divided into K long blocks and jointly encoded 
as K parallel sources, see flU for details. 

"in the high-dimensional limit, the quantization noise becomes approximately Gaussian distributed (in a divergence sense) 
1221 . However, for achievability results based on linear operations, such as the ones that appear in this paper, we do not need 
to rely on Gaussianity. 
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Fig. 14. Nested DSQ/DPCM MD encoder. 
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Fig. 15. DSQ/DPCM MD decoder. 



The encoder and decoder which materialize the optimal test channel are presented in Fig. [14 
and Fig. Q31 respectively. All of the switches in the encoder and the decoder are synchronized 
The up sampling operation followed by lowpass filtering introduces a half-sample delay on the 
odd samples. This delay is corrected at the decoder by the delay operator z^ 1 combined with 
the pair of up and downsamplers, see Fig. [151 The outputs of the quantizer blocks Q(-) are the 
reconstructed values Yi[k] and Y 2 [k). Moreover, at each time k, the codeword of quantizer 1 



It is to be understood that the switches change their positions with the upsampled rate (m). Thus, in the encoder shown in 
Fig. [SI the even samples Bi[n] of B[m] will go on the upper branch and the odd samples Ba[w] will go on the lower branch. 
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(quantizer 2) is entropy-coded (conditioned upon the dither signal) and put into packet 1 (packet 
2). The packet encoding operation is reversed at the decoder in order to obtain Yi[k] or Y 2 [k]. 

Since the two side descriptions consist of the even and odd instances of V[m], applying the 
predictor A(z) to each description in the original source rate is equivalent to the predictor A(z 2 ) 
in the test channel. The two quantizer dither sequences are taken to be mutually independent, thus 
the two ECDQs are equivalent to the additive noise Z[m] of the test channel. If each quantizer 
block is taken to be a high-dimensional ECDQ with the required rate, and Consequently, the two 
descriptions Yi[n] and Y 2 [n) are equivalent to the odd and even samples, respectively, of Y[m] 
in the equivalent channel, and finally the whole channel from the source to the central and side 
reconstructions is equivalent to the channel from X[n) to Xp[n] and Xg[n], respectively. 

Since we have seen that the mutual information in the test channel achieves (with optimized 
spectra + (e J " w ) and 0_(e J ' w )) the colored MD RDF, the encoder/decoder pair of Figs. [141 and 
[l"5l is able to achieve the complete RDF for stationary Gaussian sources at all resolutions and 
for any desired side-to-central distortion ratio. 

Remark 8. In the scheme shown in Fig. [74] the two prediction loops are embedded within a 
common noise shaping loop. Alternatively, one may alter the nesting order and let the common 
noise shaping loop be embedded within the two prediction loops. At high-resolution conditions, 
there is no loss of performance by switching the nesting order. However, at general resolution, 
the latter approach is suboptimal. The reason is, that for white quantization noise, the DPCM 
loop also shows to the outside a total white noise (by the basic DPCM equality /l23l/). while 
the DSQ loop shapes the noise. Since the DPCM loop assumes white noise for optimality /[9]/, 
it cannot be built around the shaped DSQ noise. 

VII. Conclusions and Discussion 

A parametric formulation of the two-description symmetric RDF for stationary colored Gaus- 
sian sources and MSE was presented. This result was established by providing a spectral domain 
characterization of the optimum side and central distortion spectra. For white Gaussian sources, 
the optimum distortion spectral density is a two step function. For colored sources, the optimum 
distortion spectral density is generally not piece-wise flat but depends upon the source spectral 
density and the desired resolution (i.e., the desired central and side distortion levels). It was 
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furthermore shown that the symmetric MD RDF could be achieved by a time-domain approach 
based on prediction and noise-shaping. The time domain implementation demonstrated that, at 
high resolutions, it was possible to separate the mechanism responsible for exploiting the source 
memory (DPCM) from the mechanism controlling the MD coding parameters (noise shaping). 

Appendix A 
Proof of Lemma CD 

There are three cases to consider, depending upon the sign of S(e J ^). If E^e^) > 0, then 
there is one real root and two complex roots. If H(e J ' w ) < 0, there are three real distinct roots. 
Finally, if H(e JW ) = 0, there is a single real triple root (if g(e JW ) = 0) or one real root and one 
real double root (if q(e^ UJ ) ^ 0) ETTl . Thus, for every choice of (Ai,A 2 ), one may identify the 
admissible solutions of (l45l) - (PTTl) . i.e., the ones that are inside the triangular region (|28l) (recall 
Figure |4}. 

It easy to show that the discriminant (|4T|) satisfies: 

E(en = ~ X[ 432A6 ^ -^(^))(A 2 -ff (e*"))(A a -£f (e*"))(A 2 -gf (e*")), 

(75) 

where {^{e^)} are the four real roots of H(e jaJ ) given by £f (e JW ) = 0,£f{e juJ ) = -X u 

s 2S x (e>»)*i + 85|(e^)Af - 16S x (e^)\j - 3 + Vgg^ggg + 1)3 

^ 2le j " 4S x (e^)(4£|(e^)A? + l) ' U } 



2S x (e>»)Xi + SSUe^)\l - 16S 2 x (e^)\j - 3 - 2 ^/gggggAj + 1)3 
" J> ' 45'x(e^)(4^(e^)A? + l) ' W ; 



Since we only have to consider non-negative multipliers, £f {^ w ) < 0. Clearly, { e ^) < 
£3 (e JW ),Vu;. The following lemma states the signs of ^(e?") an ^ ^(^'"O- 

Lemma 3. For Ai > 0,^f(e JW ) > 0. Moreover, the sign of £f(e ju ) ft76\) is given by: 

sign{g{en) = sign ( * - A^ . (78) 
Proof: We first show that £3 (e*") (1771) is non-negative. To do so, we show that 



2S x (e juJ )X 1 + 8S x (e j ")\l - 165|(e^)A? - 3 - 2 A /2(25^( 6 » W )A? + l) 3 < 0, (79) 
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which means that (77) is non-negative. Let ifi(e ju ) = 2S x (e ju> )X 1 + 8S x (e ju )Xl and 
(p 2 (e? u ) = 2y/2(2S%(e?")\l + l) 3 and notice that it is enough to show that (pi(e? u ) < <p 2 (e joj ), Vw. 
Since (pi(e^ u ) and <£2{& w ) are both positive functions, we may work on their squares, i.e., 
<p\(ei") = AS x (e^)X 2 1 +32S x (e^)Xf+d4S x (e^)X 6 1 and^l(e^) = 64S£(e^)A?+192££ (e^)Af+ 
192S x (e :jul )Xl + 6A. Forming the inequality <p\{e? w ) > <p\{e>") and collecting similar terms yields 
64 > -188S^(e»' w )Af - 160S^(e^)A? which is always satisfied for A x G R. This proves the 
first part of the lemma. 

We now consider the sign of £f( e5 ' a; ) Let = 2x + 8x 3 — 16x 2 — 3 and cp 2 = 

2a/2(2x 2 + l) 3 . The discriminant of <pi is strictly positive so ipx has only a single real root, 
which is located at £ = 1.96973 where we note that £ > ~. Moreover, x = =>- </?i = — 3 and it 
follows that (pi < for x < £ and > for x > £. Notice also that y? 2 > for x > 0. 

At this point we let h = iff — cp 2 = — 256(z — £)(z + — |) 3 , which is a fifth-order 
polynomial having a pair of complex conjugate roots at x — ±i/2 and a real (triple) root at 
x = 1/4. Thus, h crosses the real line only once. Since h = 1 for x = it follows that h > 
for x < 1/4 and h < for x > 1/4. Furthermore, h = for x = 1/4. 

Since h > for x < 1/4 it follows that > which implies that + <^ 2 < since 
tpi < for x < 1/4. The first case of (|78l) now follows by inserting x = AiiS f x-(e ? ° w ) in (pi 
and remembering the additional sign from (1771) . Since /i = implies that + (p 2 = 0, it 
immediately follows that sign{^ 2 {e^ w )) = for x = XiSx{e^) = 1/4. Finally, for h < we 
have </??, > Lff which implies that cp 2 > cpi since cp 2 is positive and it follows that §f (e^) < 
for AiS^e-^) > 1/4. This proves the remaining parts of the lemma. ■ 

In Fig. [16] we illustrate the possible sign behavior of H(e J ' w ) as a function of A 2 , using Lemma[3] 
and the fact that by (|73l) . lim^^-too ^(e^) = — oo, Vw. Building on this, we prove Lemma[T]by 
considering the following three cases. 

A. Negative Discriminant 

In this case E{e^) = g 2 (e JaJ ) + p 3 (e^) < and we have three real solutions. It is easy to 
see that we must have p(e jul ) < and \p(e ju )\ 3 > q 2 (e jul ). Let z^e^) = q(e jul ) + ^E(e^) = 
q(e ju ) + iy/-Z(e? a ) and z 2 (e ju ) = q(e ju> ) - y/E(e? u ) = q{e ju> ) - i^/-E(e^) and notice that 
Si(e juJ ) =f/zi(ei"),i = 1,2. Since E(e juj ),p(e j0J ) < 0, it follows that \zi{e> u )\ = \z 2 {e juJ )\ = 
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Fig. 16. The possible zero locations for the discriminant E(e J ' < ") as a function of A 2 for a given u>. We note that the £f s are 
functions of w. 



a/— p(e^ w ) 3 = A/|p(e^)| 3 . Moreover, the phase is given by 

arctan( v /-S(e^)/g(e^)), g(e JW ) > 0, 

01^) = <i tt + arctan( v /-S(e^)/g(e^)), g(e JW ) < 0, 

tt/2, g(e>" w ) = 0, 



(80) 



and 02 (e 



(e^). Thus, 0i (e^) G [0;tt]. The value of 0i(e^) depends upon g(e J ' w ), 



which is a third-order real polynomial in A 2 having a negative (or zero) discriminant. Thus, it 
has three real roots {€i(e? u )},i = 0, 1,2, which after some algebra can be shown to be given 
by 



Q(en = 7j ^|-^v / 6 V / 25|( e ^)A? + lcos 



2 2S x (ei") 



(81) 
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Cl(e j n = yV^2S* (e**)A? + 1 ^sin (^f^ + cos 

$(e*>) = j£^^ 2S x(^l + 1 1 " v^sin (^^) + cos 
1 

+ 2S^j ' (83) 

where 

= arctan (^768^-) A? + 1152^(e^)A? + 576S^(e^)Af + 15^ . (84) 

Moreover, lim^oo g(e^) = oo and lim A2 ^_ 00 g(e^) = -oo. Thus, q(e juJ ) > if A 2 > H(e juj ) 
or if £l(e? u ) < A 2 < £|(e*"). 

With this, it is easy to show that the solutions (roots), {xi(e ju) )}f =1 , as given by (l45l)-(l47l). 
can be written as 



We note that 



Xl (en = 2VTp^)| cos(^i(e* w )/3) - ^f^, (85) 
x 2 (e^) = - vlKi^)l(cos(0 1 (e'' u )/3) + v^sin^^)/^) - (86) 
ar 3 (^ w ) = - vlK^Ol(cos(0i(e 3 " w )/3) - \/3sin(0i(e^)/3)) - (87) 



min 2cos(C/3) > max -(cos(C/3) - VSsm(C/3)) (88) 

CS[0;tt] fe[0;?r] 



and that 



min —(cos (C/3) - \/3sin(C/3)) > max -(cos(C/3) + v / 3sin(C/3), (89) 

C6[0;tt] C£[0;7r] 

which implies that x\{e^) > x 3 (e JW ) > x 2 (e JW ) for any pair (Ai, A 2 ) and for all to E [— tt; it]. 

Let us first consider the solution Xs(eP u ) given by (|87l) . From Lemma |4] below, it follows that 
x^{eP u ) > Sx{e? u )/% which violates the spectral constraint. Moreover, since £i(e JW ) > x^{e^), 
we deduce that x 2 (e jul ) given by (|86l) is the only admissible candidate solution. 



September 29, 2011 



DRAFT 



SUBMITTED TO IEEE TRANSACTIONS ON INFORMATION THEORY, MAY 2010 



39 



Lemma 4. Let E(e juJ ) < 0. Then, for any positive A x and X 2 ,x 3 (e ju ') > Sx{eP u )/2,\/u, where 
Xs(e juj ) is given by rf87l) . 

Proof: Since S(e JW ) < 0, we only need to consider p^e^) < 0. Thus, \/\p(e^)\ = 
J — pTefi). Moreover, ^\Ap(e^)\ = c ^ eJ ) (^xjfM^i- 1 ) ^ f or some everywhere positive func- 

OA 2 V \/b( e: '")| 3 

tion c(e ju) ). It follows that for Ai > 2 s x \& u ) ' V\p( ejul )\ nas increasing slope in A 2 . Taking 
the limit A 2 — > oo shows that the maximum slope of \/\p(ei u )\ is Sx ^ i - ■ On the other 
hand, — a2 ^ - > 3S x {e^ U) ) /2 and is increasing in A 2 with constant slope Sx ^ ^ - Moreover, 
cos(0!(e^)/3) - v / 3sin(0 1 (e^)/3) < 1. Thus, x 3 (e^) > -y/\p{e'»)\ - ^f 1 > 3S x {e j ")/2 
for Ax > l/(2S x (ei")). 

Let us now instead assume that Ai < 1/ {2S x {e^)). We will lower-bound 6_(e- ?< ^), showing 
that this is a high-distortion case. To that end, we first equate (|37l ) to zero and isolate A 2 , which 
when inserted into (|39l) leads to: 

Q + (e> u ) = S x (e ju) ) - e(e j ") + 4 f^P- x . ( 90 ) 

+ K J K J v ; 2Ai 4Ai9_(e^) 

where we have also substituted A2 (e^) = 0_(e ja; ). Using the non-negativity of 0_(e- ?w ) 
and 0+(e JaJ ) and inserting © into e_(e*") + 6+(e JW ) > leads to 

e > S J^1 

which on behalf of the assumption A x < l/(2S x (e^)) implies that e_(e* w ) > S x (e j0J )/4, VA 2 . 
In this case -y/\p(e^)\ - a 2 (e jul )/3 is concave in A 2 . Clearly, [—y/\p{ei w )\ - a 2 /3]A 2 =o > 



Sx{e juJ )/2 and we know from above that in the limit A 2 ->■ oo, —\/\p{e ju) )\ - a 2 (e^)/3 > 
Sx{e^)/2. Thus, since we can lower bound a concave function by an affine function, it follows 
that - v /\p{e^)\-a 2 {e juj )/3 > S x {e juj )/2. Thus, x 3 (e> u ) > S x {e juJ )/2 as was to be proven. ■ 

B. Positive Discriminant 

In this case E{e^) > and we have only a single real solution given by xi(e^ u ) (1451) . 



C. Zero Discriminant 

In this case E{e^) = 0, which is possible if —p*(e? u ) = q 2 (e^ UJ ). The positive zeros of 
H(e 7U ') are given by £f (e JW ), £3 (e- 7 ^), where the former is only positive if Ai < AS ^ ej ^ ■ Since 
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q(e juJ ) 7^ when E(e juJ ) = 0, there are two real solutions i.e., x 1 (e jtJ ) and x 2 (e juJ ) = x 3 (e jui ). 
We now show that Xi(e juJ ) is the desired solution. 

Let A 2 = ^(e^) for some to. Then it is easy to show that si{e^) = s 2 {e^) < and clearly 
-a 2 (e*")/3 > S x (e? u )/2. Thus, x 2 (e^) = x 3 (e? u ) = -§(si(e*") + s 2 (e*")) - a 2 (e*")/3 > 
S x (e juJ )/2, which is not an admissible solution. 

Now let A 2 = £f(e juJ ), which is positive if and only if < A x < 1/ ' (AS x (e ju) )). Moreover, 
assume that q(e juJ ) < since otherwise x 2 (e juj ) = x 3 (e juJ ) is clearly greater than S x (e juJ )/2. We 
first show that p(e jlJ ) < for A 2 = £ 2 (e^). After some algebra, we get 



where 



and 



tpiie?") = 16v / 2y / (25|(e^)A? + lfS 2 x {e^)\\ + 2v / 2y / (2S 2 : (e^)A 2 + l) 3 (92) 



(p 2 (e> u ) = 3 + 26S 2 x (e j0J )\ 2 1 + KMA^e*") + 128S , £(e , ' w )A?. (93) 



Since (/^(e-^) and ip 2 (e ju} ) are both positive functions, we can work on their squares, and form 
the inequality y? 2 (e^) - y? 2 (e itJ ), that is 

ip 2 2 {e j ") - ipl(e ju} ) = 1 - 20S%(e? u )\l - 7Q\\S* x {e? u ) + lh<AS? x (e? u )\\ + 10304S|(e^)A* 
+ 22528Al°S'iV w ) + 16384^ 2 (e^)A} 2 , (94) 

which is clearly positive for all < Ai < 1/ \AS x (e juJ )). 

Let us now consider the zeros of E(e juJ ) = q 2 (e jlJ ) + p 3 (e jiJ ) and q 2 {e iuj ) from a geometric 
point of view. First, q(e ju) ) is a third-order polynomial and q 2 (e juJ ) is a non-negative sixth-order 
polynomial that shares zeros with q(e juJ ). Moreover, p(e juJ ) < as we established above and 
therefore p 3 (e ja; ) < for A 2 = If we consider the middle zero of g 2 (e JW ), i.e., £f(e JW ), 

then adding the negative function p 3 (e JW ) to q 2 {e^) will result in two zeros around the point 
A 2 = £f(e JW ) instead of a single zero at A 2 = £f(e JW ). In fact, the smaller of the zeros becomes 
£f(e? u ) and the larger zero becomes ff (e juJ ). But then clearly fg(e*") > £f(e^) > ff (a*") and 
it follows that q(e juj ) < for A 2 = ff (e juj ). This shows that x 2 (e juj ) = x 3 (e ju} ) > S x (e juj )/2 
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also for A 2 = ff (e ju1 )- 

Thus, the candidate solution for H(e J ' w ) = 0, is therefore given by Xx(e juJ ). 

Appendix B 
Proof of Proposition [51 

A. Case Ai > and A 2 S> 1 
In this case, we note that 

H(e-) = + ^|(e-)A?) + (95) 

It follows that E(e^) < for large A 2 . Furthermore, 

^ H « +o( |) (96) 



and 0i(e ja O = arctan ( ^ffj ) 

We use the solution X2(e? w ) given by (|46|) and need to carefully address its limiting behavior 
in A 2 , since the dominating terms cancel. The first-order Taylor approximation of arctan(x) is 
arctan(x) = x + 0(x 2 ), V|x| < 1. Thus, 



^) \_ 3^3 _ / ; T^-— 2 , „ / A? 



1 (e>") = arctan v , >. ' = AC » a/ 1 + 4S|(e^)A? + O ^4= , (97) 



g(e^) 45 x (e^)A 2 V ^ > ' \ fx 



where the approximation becomes an equality in the limit as A 2 — > oo since this implies that 
4>i(e juJ ) — >■ 0. Similarly, for all x, 

/o 2 /3 

cos(x/3) + v / 3sin(x/3) = 1 + — a; + C(x 2 ), (cos(x/3) + v / 3sin(x/3)) 2 = 1 + —x + 0{x 2 ). 

3 3 

(98) 

Let at(e? u ) = cos(0i(e J " w )/3) + v / 3sin(0 1 (e i ° J )/3). Then, we can write 



x 2 (e^) = -^(i^y|a(e^) - (99) 



|p(e^)|a(e^) 2 -^l! 



9 



(100) 
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From ([98]) and using (1971) . it follows that 

3 



a(e 3u) y = 1 + 



2S x (e^)X 2 

With this, we can write the numerator of (1 1001 ) as 



l + 4S!-(e^)A? + e> 



A? 



b(e-)l«(e-) 2 - a2( ^ 2 - 



9 



6X1 



2S x (e j ")\ 1 + 1-J1 + AS 2 x (e^)\\ + O 



(101) 



Ai 



V*2 

(102) 



On the other hand, since linix^oo «(e JtJ ) = 1, Wu, the denominator of (1 1001) can be written as 
(for large A 2 ) 



q 2 (e^) ^ 2S x (e j ")\ 2 
3 3Ai 



(103) 



Substituting (fT02l) and (fT03l) into (flOOl) yields 

1 



B-fr'-) = _(25 x (e^)A 1 + 1 - ^1 + 4^(6*" )A?) + O 



A? 



(104) 



so that 



lim 6_(e^) = lim x 2 (e^) = — 2 > S x (e^)A 1 + 1 - , /l + 4Sl(e*")A? 

A 2 -s>oo A 2 -s>oo 4Ai 



(105) 



5. Case Ai, A 2 3> 1 

We note that when assuming Ai/v^Ao" — > 0, then the results for finite Ai in S ection IB - A I remain 
valid. We rewrite (11041) as 

-(25x(e^)Ax + 1) - v/l + 45|(e^)A? 



4Ai 



9_(e^) = -r-(25jsc(e 7a ')Ai + 1 - Jl + 4S&(e**)A? 



A? 



4^(e^)A 1 



4A X ^/i + 45|(e^)A? + 2S x (e^)Ai + 1 



+ 



(25x(e^)Ax + 1) - y/l + 4S x (ej»)\l 

(106) 

A? V 



(107) 
(108) 
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where c Al ( 



e juj ) = 0(1) and lim 




l,Vw. Inserting this into (1591) yields 




4(Ai + A 2 ) + 




O 




\\ + A 2 a/A| A? + AiA 2 



A? 1 



) 



(109) 



Finally, it follows from (11081) and (11091) that 



lim 

Ai,A2— >oo 



Ai6 



1 

T' 



(110) 



and 



lim Ai6+(Ai + A 2 ) 



1 



(HI) 



4 



Ai/-^A2"^-0 
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